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ABSTRACT . 

This report describes findings from the Comprehensive School 
Reform Implementation survey conducted in spring 2000. The purpose of the 
survey was to determine the extent to which each area of comprehensive school 
reform was a focus of reform efforts across seven Midwestern states and the 
extent to which principals and teachers believed they had made progress in 
each area. In' addition, the survey prompted principals and teachers to report 
on the successes and challenges of comprehensive school reform and changes or 
improvements in teaching as a result of reform efforts. The principal or at 
least one teacher from 221 of the 361 targeted schools completed the survey. 
This report summarizes their responses. The survey findings reveal the 
following areas were the focus of reform: (1) developing a comprehensive 

reform plan; (2) obtaining staff support; (3) establishing measurable goals; 
(4) using effective research-based strategies for improving curriculum and 
instruction; and (5) providing professional development. Additionally, the 
following areas were most often identified for future efforts: using 
research-based strategies to improve integration of technology and 
instruction, and school organization and management; involving parents in 
comprehensive school reform; and using additional technical assistance from 
external sources. (Contains 15 exhibits.) ( WFA) 
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Introduction 



This report describes findings from the Comprehensive School Reform (CSR) Implementation 
Survey that was conducted by the North Central Regional Educational Laboratory (NCREL) in 
spring 2000. This survey was part of a larger study of the Comprehensive School Reform 
Demonstration (CSRD) Program undertaken by NCREL and RMC Research Corporation during 
2000 (see Faddis et al., 2000). NCREL developed the survey as a means of assessing the extent 
to which schools receiving CSRD funding had focused on each of the nine comprehensive school 
reform program components (comprehensive reform design, support within the school, 
measurable goals and benchmarks for student performance, parental and community 
involvement, effective research-based methods and strategies, professional development, 
external technical support and assistance, coordination with the school's other reform efforts, and 
evaluation strategies) and two additional components (support from the district office, and 
coordination of financial resources). NCREL also was interested in examining the progress that 
schools had made in implementing these components as part of their reform efforts. 

NCREL mailed principal surveys to 361 schools in seven states in the North Central region of 
the United States. In addition, NCREL asked each of the 361 principals to distribute the teacher 
survey to two teachers engaged in the comprehensive school reform program at his or her school. 
The first part of the survey asked the principals and teachers to rate whether each reform 
component was a past, present (secondary or primary), or future focus. Respondents were then 
asked to rate the level of progress achieved for each area of focus. Respondents rated their 
progress as “goals achieved” or indicated their degree of progress on a scale from 1 (no progress 
made ) to 4 (significant progress made). The second part of the survey asked principals and 
teachers one of three open-ended questions regarding important successes, significant challenges, 
or changes or improvements in teaching. The evaluators selected groups of schools to respond to 
each question. Copies of the principal and teacher surveys are in the Appendix. 

This report summarizes the principal and teacher survey responses. The first section reports the 
survey response rates of principals and teachers across the seven states. The second section 
provides additional sample characteristics of the schools that responded to the surveys. 
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Following these general descriptive data are the results of analyses of the principal and teacher 
survey data at the aggregate and school levels. In addition to mailing surveys to the 361 schools, 
the evaluators obtained school information from the Southwest Educational Development 
Laboratory’s (SEDL) national database of schools receiving CSRD grants. The SEDL database 
provided complete data on geographic locale (urban, suburban, or rural), grade level (elementary, 
middle, high school, or combined), and Title I status (schoolwide, targeted assistance, yes 
[unspecified type], or no Title I) for 93 percent of the sample schools. The final component of 
the quantitative data analysis involved cross-group comparisons — that is, comparison of school 
differences across geographic locale, grade level, and Title I status. Responses to the open-ended 
survey questions are discussed in the final section of this report. 

Response Rates by State 

Of the 361 schools that received surveys, the principal or at least one teacher from 221 schools 
(61 percent) completed and returned a survey. Exhibit 1 shows the principal and teacher survey 
response rates for each of the seven states. The principal return rate across the seven states was 
58 percent, and the teacher return rate across all states was 54 percent. Indiana produced the 
highest principal and teacher return rates with 89 percent of schools returning principal surveys 
and 83 percent of schools returning at least one teacher survey. Approximately three-fourths of 
the targeted schools in Wisconsin also returned principal and teacher surveys. Illinois produced 
the lowest rate of the seven states with 47 percent of the targeted schools returning principal 
surveys, and 35 percent of the targeted schools returning teacher surveys. 
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Exhibit 1 

CSR Implementation Survey Response Rate by State 



State 


Survey 

Type 


No. Schools 
Targeted 


No. Schools 
Responding 


Response 

Rate 


IL 


P 


109 


51 


47% 




T 


109 


38 


35% 


IN 


P 


18 


16 


89% 




T 


18 


15 


83% 


IA 


P 


16 


11 


69% 




T 


16 


10 


63% 


MI 


P 


73 


44 


60% 




T 


73 


45 


62% 


MN 


P 


23 


15 


65% 




T 


23 


17 


74% 


OH 


P 


88 


46 


52% 




T 


88 


44 


50% 


WI 


P 


34 


26 


76% 




T 


34 


25 


74% 


Total 


P 


361 


209 


58% 




T 


361 


194 


54% 


Note. P 


= principal, T 


= teacher. 







Exhibit 2 shows the school response rates by grade level, geographic locale, and Title I status. 
Elementary schools had a higher response rate than middle or high schools; suburban and rural 
schools had a higher response rate than urban schools; and targeted assistance and unspecified 
Title I program types had a higher response rate than schoolwide Title I or non-Title I schools. 
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Exhibit 2 

CSR Implementation Survey Response Rate by Grade Level, Geographic Locale, and 

Title I Status 



Characteristic 


Mailed 


Returned 


Percent 


Grade level 


Elementary 


236 


149 


63% 


Middle 


52 


28 


54% 


High 


55 


28 


51% 


Combined 


7 


7 


100% 


Geographic locale 


Urban 


226 


126 


56% 


Suburban 


66 


48 


73% 


Rural 


53 


37 


70% 


Title I status 


Schoolwide 


206 


120 


58% 


Targeted assistance 


49 


35 


71% 


Yes (unspecified) 


43 


31 


72% 


No 


51 


26 


51% 



Sample Characteristics 

Sixty-nine percent of the responding schools (153 of 221) returned completed surveys from the 
principal and two teachers. Thirteen percent of the responding schools returned surveys from the 
principal and one teacher, and 12 percent returned surveys from the principal only. The 
remaining 6 percent of responding schools returned no principal surveys but returned a survey 
from at least one teacher. Schools did not differ on dimensions of geographic locale, grade level, 
or Title I status according to whether one, two, or three individuals returned the survey. 

To assess the overall representativeness of the returned survey sample, the evaluators compared 
the schools that returned at least one survey to schools that did not return any surveys on the 
following dimensions: geographic locale, grade level, and Title I status. A Chi-square test 
revealed that urban schools were less represented in the returnee sample (60 percent) than in the 
nonretumee sample (75 percent), % 2 (2, n - 345) — 8.17,/? — .05. Tests also revealed that 
elementary schools were slightly more represented in the returnee sample (70 percent) than in the 
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nonretumee sample (63 percent), % 2 (3, n = 350) = 8.34, p < .05. Returnee schools were 
equivalent to the nonretumee schools on Title I status. 

The SEDL database reported complete data for 203 of the 221 schools that returned at least one 
CSR Implementation Survey. Exhibit 3 shows the number and percentage of respondent schools 
at each grade level by geographic locale and Title I status. 



Exhibit 3 

Number of Respondent Schools by Grade Level, Geographic Locale, and Title I Status 



Characteristic 


Elementary 
(if = 143) 


Middle 
(n = 27) 


High 

(j n = 26) 


Combined 

(/i = 7) 


Geographic locale 


Urban 


87 (61%) 


13 (48%) 


19 (73%) 


3 


(43%) 


Suburban 


34 (24%) 


8 (30%) 


3 (12%) 


0 


(0%) 


Rural 


22 (15%) 


6 (22%) 


4 (15%) 


4 


(57%) 


Title I status 


Schoolwide 


94 (66%) 


11 (41%) 


10 (38%) 


1 


(14%) 


Targeted assistance 


19 (13%) 


9 (33%) 


1 (4%) 


3 


(43%) 


Yes (unspecified) 


22 (15%) 


2 (7%) 


4 (15%) 


1 


(14%) 


No 


8 (6%) 


5 (19%) 


1 1 (42%) 


2 


(29%) 



Of the 144 elementary schools that returned surveys, the majority (61 percent) were located in 
urban areas, 24 percent were located in suburban areas, and 15 percent were located in rural 
regions. This pattern is similar for middle schools and high schools. The percentages of schools 
returning surveys in urban, suburban, and rural regions were 48 percent, 30 percent, and 22 
percent, respectively, for middle schools (« = 27), and 73 percent, 12 percent, and 15 percent for 
high schools (« = 28). Three of the seven combined schools that returned surveys were in urban 
areas, and the remaining four were in rural areas. 

The majority (66 percent) of the elementary schools that returned surveys were Title I 
schoolwide status. Middle and high schools also were more likely to be Title I schoolwide than 
targeted-assistance Title I. Elementary schools and middle schools were more likely than high 
schools to be receiving any type of Title I funding. 
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Areas of Comprehensive School Reform Focus 



To assess schools’ implementation status for each of the reform components, the survey asked 
principals and teachers to report whether each component was a past, present, or future focus. 
Exhibit 4 depicts the percentage of all principals and all teachers who reported each reform 
component as a past, present, or future focus. 

Exhibit 4 

Principal and Teacher Report of Status of Reform Component Foci 



Principals Teachers 

( n = 209) (n = 355) 



Reform Component 


Past 


Present 


Future 


Past 


Present 


Future 


Developing a comprehensive school reform plan 
covering nine components that are appropriately 
aligned 


0% 


99% 


1% 


0% 


98% 


1% 


Obtaining support from entire staff in 
implementing reform models 


3% 


94% 


3% 


3% 


94% 


3% 


Establishing measurable goals and benchmarks 
for student achievement that are aligned with 
state standards 


3% 


95% 


2% 


3% 


94% 


3% 


Involving parents in comprehensive school 
reform 


1% 


87% 


13% 


0% 


83% 


17% 


Using effective, research-based strategies for: 














Improving curriculum 


1% 


95% 


4% 


1% 


94% 


5% 


Improving instructional strategies 


0% 


99% 


1% 


0% 


98% 


2% 


Improving integration of technology and 
instruction 


1% 


80% 


19% 


1% 


78% 


20% 


Improving school organization or management 


4% 


85% 


11% 


3% 


83% 


14% 


Improving assessment of student progress 


1% 


94% 


5% 


1% 


93% 


7% 


Providing professional development for teachers 
to help implement reform strategies 


0% 


98% 


2% 


0% 


97% 


3% 


Obtaining additional technical assistance from 
external sources 


1% 


83% 


16% 


1% 


81% 


18% 


Using data to evaluate implementation and 
outcomes of comprehensive school reform efforts 


1% 


91% 


8% 


1% 


90% 


10% 


Obtaining support from district office to 
implement reform strategies 


3% 


86% 


11% 


2% 


85% 


13% 



(exhibit continues) 
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Principals 
(n = 209) 






Teachers 
( n = 355) 




Reform Component 


Past 


Present 


Future 


Past 


Present 


Future 


Ensuring that the CSRD-fiinded reform effort is 
coordinated with the school’s other reform efforts 


2% 


92% 


6% 


1% 


91% 


8% 


Coordinating financial resources for 


1% 


93% 


6% 


1% 


91% 


8% 



comprehensive school reform with other 
resources to support reform 

In general, the majority of principals and teachers responded similarly that each of the identified 
reform components was a present focus of their school reform (between 90 percent and 1 00 
percent of the teachers and principals reported 1 0 of the 15 components to be a present focus). 
Areas that survey respondents most often identified as future foci, however, were the use of 
research-based strategies to improve the integration of technology and instruction (19 percent of 
the principals and 20 percent of the teachers identified this component as a future focus), the use 
of research-based strategies to improve school organization or management (identified by 1 1 
percent of the principals and 14 percent of the teachers), the involvement of parents in 
comprehensive school reform (identified by 13 percent of the principals and 17 percent of the 
teachers), obtaining support from the district office to implement reform strategies (identified by 
1 1 percent of the principals and 1 3 percent of the teachers), and the use of additional technical 
assistance from external sources (identified by 1 6 percent of the principals and 1 8 percent of the 
teachers as a future focus). 

As reflected by the high numbers of principals and teachers who identified each reform 
component as a present focus, the average number of present foci reported by principals and 
teachers also was high. The average number of present foci reported by principals was 1 3.9 out 
of a possible 15, and the average number of present foci reported by teachers was 13.3. The 
survey also asked respondents to report whether present foci were primary or secondary. Broken 
down by primary and secondary foci, the average numbers of primary foci reported by principals 
and teachers were 10.4 and 9.5, respectively. The average numbers of secondary foci reported by 
principals and teachers were 3.6 and 3.8, respectively. 

Exhibit 5 categorizes each present focus as a primary or secondary focus and shows the 
percentage of the principals and teachers who identified each of the reform components to be 
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either a primary or secondary focus at their school. It also is noteworthy that data presented in 
Exhibit 5 generally support the data presented in Exhibit A — that is, the reform components that 
principals and teachers most identified as secondary foci in Exhibit 5 were the same components 
they identified as future foci in Exhibit 4. 



Exhibit 5 

Principal and Teacher Report of Reform Component Focus Level 



Primary Focus Secondary Focus 



Reform Component 


Principal 


Teacher 


Principal 


Teacher 


Developing a comprehensive school reform plan 
covering nine components that are appropriately aligned 


96% 


91% 


4% 


9% 


Obtaining support from entire staff in implementing 
reform models 


91% 


84% 


9% 


16% 


Establishing measurable goals and benchmarks for 
student achievement that are aligned with state standards 


86% 


80% 


14% 


20% 


Involving parents in comprehensive school reform 
Using effective, research-based strategies for: 


57% 


57% 


43% 


43% 


Improving curriculum 


85% 


84% 


16% 


16% 


Improving instructional strategies 


90% 


86% 


10% 


14% 


Improving integration of technology and instruction 


46% 


48% 


54% 


52% 


Improving school organization or management 


56% 


52% 


44% 


48% 


Improving assessment of student progress 


75% 


76% 


25% 


24% 


Providing professional development for teachers to help 
implement reform strategies 


93% 


84% 


7% 


16% 


Obtaining additional technical assistance from external 
sources 


55% 


58% 


45% 


42% 


Using data to evaluate implementation and outcomes of 
comprehensive school reform efforts 


72% 


66% 


28% 


34% 


Obtaining support from district office to implement 
reform strategies 


53% 


52% 


47% 


48% 


Ensuring that the CSRD-funded reform effort is 
coordinated with the school’s other reform efforts 


82% 


71% 


18% 


39% 


Coordinating financial resources for comprehensive 
school reform with other resources to support reform 


70% 


67% 


30% 


33% 



Note. Principal subsample sizes per item ranged from 167 to 207, Teacher subsample sizes per item ranged from 277 to 347. 
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The principal and teacher reports were generally similar, with the exception that principals (82 
percent) were more likely than teachers (71 percent) to report that coordinating comprehensive 
school reform with other reform efforts was a primary, as opposed to a secondary, goal. Overall, 
more than 80 percent of the principals and teachers reported these six areas to be the primary foci 
at their schools: 

• Developing a comprehensive reform plan. 

• Obtaining staff support. 

• Establishing measurable goals and benchmarks. 

• Using effective, research-based strategies for improving curriculum. 

• Using effective, research-based strategies for improving instruction. 

• Providing professional development. 
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Progress Achieving Reform Goals 



To assess schools’ progress in achieving the goals of each of the reform components, the survey 
asked principals and teachers to rate the progress they had made toward goal achievement. The 
evaluators subsequently created two dimensions of goal achievement: goal was achieved (yes or 
no); and, if the goal was not achieved, the degree of progress made toward it (1 = no progress 
made, 4 = significant progress made). Exhibit 6 shows the percentage of principals and teachers 
in the sample who reported that their school already had achieved the goals that they intended to 
accomplish. 



Exhibit 6 

Principal and Teacher Report of Schools’ Reform Component Goal Achievement 



Reform Component 


Principals 


Teachers 


Developing a comprehensive school reform plan covering 
nine components that are appropriately aligned 


10% 


10% 


Obtaining support from entire staff in implementing 
reform models 


24% 


16% 


Establishing measurable goals and benchmarks for student 
achievement that are aligned with state standards 


19% 


21% 


Involving parents in comprehensive school reform 
Using effective, research-based strategies for: 


8% 


6% 


Improving curriculum 


8% 


11% 


Improving instructional strategies 


8% 


10% 


Improving integration of technology and instruction 


7% 


9% 


Improving school organization or management 


11% 


7% 


Improving assessment of student progress 


10% 


10% 


Providing professional development for teachers to help 
implement reform strategies 


26% 


23% 


Obtaining additional technical assistance from external 
sources 


17% 


14% 


Using data to evaluate implementation and outcomes of 
comprehensive school reform efforts 


10% 


15% 


Obtaining support from district office to implement reform 
strategies 


19% 


16% 


Ensuring that the CSRD-funded reform effort is 
coordinated with the school’s other reform efforts 


33% 


22% 


Coordinating financial resources for comprehensive school 
reform with other resources to support reform 


33% 


24% 


Note. Principal ws per item ranged from 194 to 209. Teacher ws per item ranged from 324 to 348. 
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For each of the 1 5 reform components, less than a third of the respondents reported that their 
school had achieved its goals. Percentages ranged from 7 percent to 33 percent of principals and 
6 percent to 24 percent of teachers. The reform areas for which at least 20 percent of the 
principals and 20 percent of the teachers reported that the schools had achieved their goals were 
professional development, the coordination of comprehensive school reform with the school’s 
other reform efforts, and the coordination of financial resources to support comprehensive school 
reform. 

With the exception of three reform components, the proportions of principals and teachers 
reporting that the schools had achieved their goals were similar. Principals were more likely than 
the teachers to report goal achievement in obtaining staff support (24 percent versus 16 percent), 
ensuring the coordination of comprehensive school reform efforts with other reform efforts at the 
school (33 percent versus 22 percent), and the coordination of financial resources for 
comprehensive school reform with other resources (33 percent versus 24 percent). 

Principals and teachers rated their schools’ progress toward the unmet goals of each reform 
component on a four-point scale (1 = no progress, 2= a little progress, 3 = some progress, and 
4 = significant progress). Exhibit 7 reports the mean ratings and standard deviations for the 
principals’ and teachers’ ratings of progress for each reform component. 
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Exhibit 7 

Comparison of Principal and Teacher Ratings 
of Schools’ Progress on Reform Components 



Reform Component 


Principals 
M SD 


Teachers 
M SD 


t 


Developing a comprehensive school reform plan 
covering nine components that are appropriately aligned 


3.77 


0.48 


3.67 


0.55 


2.07* 


Obtaining support from entire staff in implementing 
reform models 


3.78 


0.46 


3.47 


0.66 


5.94*** 


Establishing measurable goals and benchmarks for 
student achievement that are aligned with state standards 


3.56 


0.58 


3.50 


0.65 


0.96 


Involving parents in comprehensive school reform 
Using effective, research-based strategies for: 


3.06 


0.76 


2.91 


0.91 


2.05* 


Improving curriculum 


3.57 


0.68 


3.45 


0.78 


1.72 


Improving instructional strategies 


3.65 


0.54 


3.49 


0.65 


2.96** 


Improving integration of technology and instruction 


3.00 


0.84 


2.91 


0.91 


1.14 


Improving school organization or management 


3.43 


0.74 


3.19 


0.87 


3.32*** 


Improving assessment of student progress 


3.40 


0.66 


3.34 


0.80 


1.03 


Providing professional development for teachers to help 
implement reform strategies 


3.75 


0.51 


3.57 


0.66 


3.07** 


Obtaining additional technical assistance from external 
sources 


3.26 


0.86 


3.11 


0.96 


1.66 


Using data to evaluate implementation and outcomes of 
comprehensive school reform efforts 


3.41 


0.68 


3.19 


0.84 


2.95** 


Obtaining support from district office to implement 
reform strategies 


3.23 


0.88 


2.98 


0.94 


2.80** 


Ensuring that the CSRD-funded reform effort is 
coordinated with the school’s other reform efforts 


3.63 


0.63 


3.35 


0.85 


3.74*** 


Coordinating financial resources for comprehensive 
school reform with other resources to support reform 


3.67 


0.56 


3.37 


0.85 


4.15*** 



Note. Principal ns per item ranged from 135 to 188. Teacher ns per item ranged from 249 to 322. 
*p < .05. **p < .01. ***p < .001. 



In general, the principals and teachers reported moderately high ratings of the schools’ progress 
on the reform components. (Both sources’ ratings were between 3 and 4 on the four-point scale 
for most response categories.) The three areas in which schools reportedly made the greatest 
progress were developing a comprehensive reform plan with nine aligned components 
(Ms = 3.77 and 3.67 for principals and teachers, respectively), attaining staff support (Ms = 3.78 
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and 3.47 for principals and teachers, respectively), and providing professional development 
activities (Ms = 3.75 and 3.57 for principals and teachers, respectively). Conversely, the three 
areas in which principals and teachers reported making the least progress were involving parents 
(Ms = 3.06 and 2.91 for principals and teachers, respectively), improving the integration of 
technology and instruction (Ms = 3.00 and 2.91 for principals and teachers, respectively), and 
obtaining support from the district office to implement reform strategies (Ms = 3.23 and 2.98 for 
principals and teachers, respectively). Again, principals and teachers most often reported these 
reform components as future foci. 

Overall, principals’ ratings of progress were significantly higher than teachers’ ratings on 10 of 
the 15 reform components. The most significant discrepancies existed for the following four 
components: attaining staff support (Mdifference = .31); coordinating comprehensive school 
reform resources with other financial resources to support reform (Mdifference = .30); 
coordinating comprehensive school reform with other reform efforts (Mdifference = .28); and 
using effective, research-based strategies to improve school organization or management (M 
difference = .24). 



14 The Focus and Progress of Schools’ Work in Seven Midwest States 



North Central Regional Educational Laboratory 



Comparison of Principal and Teacher Ratings 



This section presents the survey results at the school level, comparing the within-school principal 
and teacher reports on their school’s degree of focus on and progress toward the goals associated 
with each reform component. These comparisons include only those respondents who identified 
the component as a present focus. For the comparison of the principals’ and teachers’ responses, 
the evaluators grouped schools that returned only one teacher survey with schools that returned 
two teacher surveys. In other words, perfect principal-teacher agreement might reflect perfect 
agreement between a single teacher and the principal or between two teachers and the principal. 

Exhibit 8 shows that the four reform components for which principals and teachers were most 
likely to reach complete agreement regarding their school’s degree of focus were: 

• Developing a comprehensive reform plan. 

• Using effective, research-based strategies to improve instructional strategies. 

• Providing professional development. 

• Obtaining staff support. 

The three reform components for which the teachers and principals in at least 25 percent of the 
responding schools revealed complete disagreement were: 

• Involving parents 

• Coordinating comprehensive school reform resources with other resources 

• Obtaining technical assistance from external sources. 

Exhibit 9 shows that principals’ and teachers’ ratings of progress toward the reform component 
goals were generally within one point of each other, indicating that there was significant 
agreement between the two sources. 
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Exhibit 9 

Alignment of Principal and Teacher Ratings of Progress on Reform Components 
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Combined Teachers’ Rating Was: 

All Sources 0.5 Points Off 1 Point Off 1.5 Points Off 2 Points Off 2.5 Points Off 3 Points Off 
in Perfect Principal’s Principal’s Principal’s Principal’s Principal’s Principal’s 
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Cross-Group Comparisons 



To determine whether response patterns existed across schools by geographic locale, grade level, 
or Title I status, the evaluators ran cross-group comparisons for each dimension with respect to 
principals’ responses as well as teachers’ responses. Specifically, the evaluators conducted one- 
way ANOVAs on present foci only. (Cell sizes would be too small to detect differences for 
either past or future foci.) Exhibit 10 compares the principals’ mean ratings of their school’s 
level of focus on each reform component across geographic locale. 



Exhibit 10 

Comparison of Principals’ Mean Ratings of Reform 
Component Focus Across Geographic Locale 



Reform Component 


Urban 
(/i = 119) 


Suburban 
(/i = 44) 


Rural 

(« = 36) 


F 


Developing a comprehensive school reform 
plan covering nine components that are 
appropriately aligned 


0.99 


0.91 


0.92 


4.07* 


Obtaining support from entire staff in 
implementing reform models 


0.94 


0.93 


0.85 


1.34 


Establishing measurable goals and benchmarks 
for student achievement that are aligned with 
state standards 


0.85 


0.90 


0.83 


0.39 


Involving parents in comprehensive school 
reform 


0.64 


0.38 


0.56 


4.12 * 


Using effective, research-based strategies for: 










Improving curriculum 


0.88 


0.77 


0.88 


1.64 


Improving instructional strategies 


0.96 


0.89 


0.69 


11.61*** 


Improving integration of technology and 
instruction 


0.49 


0.32 


0.61 


2.54 


Improving school organization or 
management 


0.63 


0.54 


0.42 


2.37 


Improving assessment of student progress 


0.81 


0.74 


0.56 


4.42* 


Providing professional development for 
teachers to help implement reform strategies 


0.95 


0.95 


0.86 


1.92 


Obtaining additional technical assistance from 
external sources 


0.59 


0.36 


0.57 


2.90 


(exhibit continues) 
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Reform Component 


Urban 
(n = 119) 


Suburban 
(/i = 44) 


Rural 
(« = 36) 


F 


Using data to evaluate implementation and 
outcomes of comprehensive school reform 
efforts 


0.77 


0.62 


0.61 


2.88 


Obtaining support from district office to 
implement reform strategies 


0.56 


0.53 


0.44 


0.61 


Ensuring that the CSRD-funded reform effort is 
coordinated with the school’s other reform 
efforts 


0.85 


0.83 


0.67 


2.99 


Coordinating financial resources for 
comprehensive school reform with other 
resources to support reform 


0.82 


0.57 


0.53 


9.03*** 


Note. 0 = secondary, 1 = primary. 
*p< .05. ***p < .001. 



The following differences were evident across the urban, suburban, and rural schools: 

■ Principals from urban schools were more likely than principals from suburban or rural 
schools to report the development of a comprehensive reform plan as a primary, as 
opposed to a secondary, focus, F(2,197) = 4.07,/? < .05. 

■ Principals from urban schools were more likely than principals from suburban or rural 
schools to report the coordination of financial resources as a primary, as opposed to a 
secondary, focus, F( 2, 1 89) = 9.03, p < .00 1 . 

■ Principals from suburban schools were more likely than principals from urban or rural 
schools to report parent involvement as a secondary, as opposed to a primary, focus, 
F(2,185) = 4.12,/? < .05. 

■ Principals from urban and suburban schools were more likely than principals from rural 
schools to report the use of effective, research-based strategies to improve instructional 
strategies as a primary, as opposed to a secondary, focus, F( 2, 197)= 1 1 .6 1 , p < .00 1 . 

■ Principals from rural schools were more likely than principals from urban or suburban 
schools to report the use of effective, research-based strategies to improve assessment of 
student progress as a secondary, as opposed to a primary, focus, F(2,189) = 4.42,/? < .05. 
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Comparisons of principals’ mean ratings of their school’s degree of focus on each reform 
component across grade levels revealed only one difference: 

■ Principals from elementary schools were more likely than principals from other schools 
to report the use of effective, research-based strategies to improve the integration of 
technology into the classroom as a secondary focus, F(3,166) = 3.27, p < .05. 

Comparisons of principals’ mean ratings of their school’s degree of focus on each reform 
component across Title I status revealed two differences: 

■ Principals from schools receiving Title I schoolwide funding and schools receiving no 
Title I funding were more likely than principals from schools receiving targeted- 
assistance Title I funding to report the use of effective, research-based strategies to 
improve school organization as a primary focus, F(3,178) = 5.84,/? < .001. 

■ Principals from schools receiving Title I schoolwide funding were more likely than 
principals from schools receiving no Title I funding to report the coordination of financial 
resources to support reform as a primary focus, F(3, 1 89) = 4.90, p < .0 1 . 

Comparisons of teachers’ mean ratings of their school’s degree of focus on each reform 
component across geographic locale, grade level, and Title I status revealed no differences. 

Cross-Group Comparisons of Progress 

Comparisons of principals’ mean ratings of their school’s progress toward the goals associated 
with each reform component across geographic locale, grade level, and Title I status revealed no 
significant differences. Similarly, comparisons of teachers’ mean ratings of their school’s 
progress toward the goals associated with each reform component across geographic locale and 
Title I status revealed no significant differences. The teachers’ responses, however, differed 
across grade level for the parent involvement reform component. Specifically, teachers from 
elementary schools reported significantly less progress involving parents in school reform than 
did teachers from schools of other grade levels, F(3,179) = 3.63,/? < .01. 
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Reported Outcomes of Comprehensive School Reform 



Each principal and teacher survey included an open-ended question about one of the following 
comprehensive school reform topics: important successes, significant challenges, and changes or 
improvements in teaching. 

Comprehensive School Reform Successes 

Exhibits 1 1 and 12 summarize the responses of principals and teachers, respectively, to open- 
ended questions about the important successes of their comprehensive school reform efforts. A 
total of 80 principals and 1 1 1 teachers responded to these questions. The successes most 
frequently mentioned by both principals and teachers were professional development and 
training, new or improved teaching strategies, and staff involvement and commitment to 
common goals or philosophy. In addition, principals commonly mentioned improved test scores 
or achievement, improved parent involvement, and staff coordination of efforts to improve 
instruction (see Exhibit 1 1). Teachers frequently mentioned the use of data to reteach skills or 
individualize instruction, the implementation of new instructional programs, staff coordination of 
efforts to improve instruction, and student progress in targeted subject areas (see Exhibit 12). 
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Exhibit 11 

Principal Report of Most Important Successes of Comprehensive School Reform 



Area of Success 


No. Principals 
Reporting 


Percent 


Staff involvement or commitment to common goals or philosophy 


26 


33% 


Professional development or training 


24 


30% 


New or improved teaching strategies 


20 


25% 


Improved test scores or achievement 


15 


19% 


Improved parent involvement 


14 


18% 


Staff coordination of efforts to improve instruction 


9 


11% 


Use of data to reteach skills or individualize instruction 


6 


8% 


Improved assessment of student progress 


6 


8% 


Improved school organization or leadership 


6 


8% 


Improved school culture or climate 


5 


6% 


Increased use of technology 


5 


6% 


Implementation of new instructional programs 


4 


5% 


Student progress in targeted subject areas 


4 


5% 


Alignment of curriculum within or across grade levels 


4 


5% 


Effective external support 


4 


5% 



Note. Multiple responses were possible, n - 80. 
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Exhibit 12 

Teacher Report of Most Important Successes of Comprehensive School Reform 



Area of Success 


No. Teachers 
Reporting 


Percent 


Professional development or training 


32 


29% 


New or improved teaching strategies 


23 


21% 


Staff involvement or commitment to common goals or philosophy 


16 


14% 


Use of data to reteach skills or individualize instruction 


12 


11% 


Implementation of new instructional programs 


11 


10% 


Staff coordination of efforts to improve instruction 


10 


10% 


Student progress in targeted subject areas 


10 


10% 


Alignment of curriculum within or across grade levels 


9 


8% 


Improved test scores 


8 


7% 


Alignment of curriculum with state standards 


7 


6% 


Improved student participation 


7 


6% 


Money for materials and equipment needed to meet goals 


6 


5% 


On-site instructional guide or coach 


6 


5% 


Improved parent involvement 


5 


5% 


Curriculum development 


4 


4% 


Time for teachers to work together 


4 


4% 


Helping students to meet goals 


4 


4% 


Effective external support 


4 


4% 



Note. Multiple responses were possible. n= 111. 



Challenges of Comprehensive School Reform 

A total of 51 principals and 83 teachers reported significant challenges to implementing 
comprehensive school reform efforts. Exhibits 13 and 14 summarize the most common 
challenges mentioned by the principals and teachers, respectively. The challenge most frequently 
identified by both principals and teachers was the extra time required to implement new 
strategies. Principals and teachers also cited the time needed for collaborating and planning with 
others and the difficulties involving parents as significant challenges to reform implementation. 
Principals reported the time required for professional development as an additional challenge. 
Challenges commonly identified by teachers included difficulty convincing staff to buy in to the 
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extra work required by reform, the reluctance of some staff to change, and implementation of all 
aspects of the reform plan. 



Exhibit 13 

Principal Report of Most Significant Challenges of Comprehensive School Reform 

No. Principals 

Challenge Reporting Percent 

Extra time required to implement new strategies 1 5 29 

Time to collaborate and plan with others 10 20 

Time for professional development 10 20 

Involving parents 8 1 6 

Measuring student progress 6 12 

Getting staff to buy in to the extra work 5 1 0 

Reluctance of some staff to change 5 10 

Staff turnover, resulting in new staff who need training 5 10 

Inadequate physical facilities 5 10 

Note. Multiple responses were possible, n = 51. 

Exhibit 14 

Teacher Report of Most Significant Challenges of Comprehensive School Reform 

No. Teachers 

Challenge Reporting Percent 

Extra time required to implement new strategies 20 24% 

Getting staff to buy in to the extra work 15 1 8% 

Reluctance of some staff to change 9 11% 

Involving parents 9 11% 

Implementing all aspects of the plan or program 8 10% 

Time to collaborate and plan with others 8 10% 

Time for professional development 6 7% 

Getting district support 6 7% 

Staff turnover, resulting in new staff who need training 5 6% 

Getting sufficient materials or software licenses 5 6% 

Measuring student progress 4 5% 

Note. Multiple responses were possible, n = 83. 



o 

ERLC 
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Changes or Improvements in Teaching 

A total of 1 10 teachers responded to open-ended questions regarding changes or improvements 
they had made in their teaching as a result of comprehensive school reform. Exhibit 15 
summarizes the most frequently mentioned changes. Nearly a third of the respondents cited the 
use of new teaching strategies. Other commonly mentioned improvements were changes in 
teaching content prompted by a greater awareness of standards and benchmarks, adoption of a 
more student-centered approach, improved organization and instructional focus, and increased 
coordination and sharing among staff. 



Exhibit 15 

Changes or Improvements in Teaching Reported by Teachers 



Improvement 


No. Teachers 
Reporting 


Percent 


Use of new teaching strategies 


35 


32% 


Increased awareness of standards and benchmarks and 
corresponding changes to lesson plans 


11 


10% 


Adoption of a more student-centered approach 


9 


8% 


Improved organization and instructional focus 


8 


7% 


Increased coordination and sharing among staff 


8 


1 % 


Use of test data to track student progress 


7 


6% 


Use of test data to identify instructional needs 


6 


5% 


Increased coaching and encouragement among teachers 


6 


5% 


Increased incorporation of real-world experiences 


5 


5% 


Increased student responsibility for learning 


5 


5% 


Greater awareness of student needs 


5 


5% 


Use of test data to group students for instruction 


4 


4% 


Use of more curriculum resources 


4 


4% 


Increased integration of different subjects 


4 


4% 



Note. Multiple responses were possible, n = 1 10. 
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Summary of Survey Findings 



The purpose of the CSR Implementation Survey was to determine the extent to which each area 
of comprehensive school reform was a focus of reform efforts across the region and the extent to 
which principals and teachers believed they had made progress in each area. In addition, the 
survey prompted principals and teachers to report on the successes and challenges of 
comprehensive school reform and changes or improvements in teaching as a result of reform 
efforts. 

The principal or at least one teacher from 221 (61 percent) of the 361 targeted schools completed 
the survey. In the majority of these schools (69 percent), the principal and two teachers returned 
surveys. Compared to the entire population of CSRD schools in the region, urban schools were 
underrepresented and elementary schools were overrepresented in the returnee sample. 

Between 90 percent and 100 percent of the principals and teachers who responded to the survey 
reported that 10 of the 15 comprehensive school reform components were a current focus of their 
reform efforts. Six areas stood out as the primary foci of reform: 

• Developing a comprehensive reform plan. 

• Obtaining staff support. 

• Establishing measurable goals and benchmarks. 

• Using effective, research-based strategies for improving curriculum. 

• Using effective, research-based strategies for improving instruction. 

• Providing professional development. 

The areas most often identified as future foci were: 

• Using research-based strategies to improve integration of technology and instruction. 

• Using research-based strategies to improve school organization and management. 

• Involving parents in comprehensive school reform. 

• Using additional technical assistance from external sources. 

The evaluators compared principal reports with teacher reports within schools on degree of focus 
on and progress toward the goals associated with each reform component. Principals and 
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teachers were most likely to agree on their school’s degree of focus on developing a 
comprehensive reform plan; using effective, research-based strategies to improve instructional 
strategies; providing professional development activities; and obtaining staff support. Principals 
and teachers were least likely to agree on the degree of focus on involving parents, coordinating 
comprehensive school reform resources with other resources, and obtaining technical assistance 
from external sources. 

Principals and teachers were generally within one point of each other in their ratings of progress 
reaching reform goals, although principal ratings of progress were noticeably higher than teacher 
ratings on 10 of the 15 reform components. The largest discrepancies occurred for attaining staff 
support, coordinating financial resources, coordinating comprehensive school reform with other 
reform efforts, and using effective research-based strategies to improve school organization or 
management. 

The evaluators also compared principal-response patterns across schools by geographic locale, 
grade level, and Title I status. For most reform components, no differences in the mean degree of 
focus were evident across these dimensions. Principals from urban schools, however, were more 
likely than other principals to report the development of a comprehensive reform plan and the 
coordination of financial resources as primary, as opposed to secondary, foci. Principals from 
suburban schools more often reported improving instructional strategies as a primary focus and 
involving parents as a secondary focus compared to other principals. Principals from rural 
schools were more likely to report improving assessment of student progress as a secondary 
rather than a primary focus. Elementary school principals tended to report improving the 
integration of technology as a secondary rather than a primary focus. Principals from schools 
receiving Title I schoolwide funding were more likely than other principals to report improving 
school organization and coordinating financial resources as primary foci. Comparisons of 
teachers’ mean ratings across geographic locale, grade level, and Title I status revealed no 
differences. 

Comparisons of principals’ mean ratings of the schools’ progress toward the goals associated 
with each reform component revealed no differences across geographic locale, grade level, or 
Title I status. Similarly, comparisons of teachers’ mean ratings of progress showed no 
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differences except for the area of involving parents. Specifically, elementary school teachers 
reported less progress involving parents in school reform than did other teachers. 

Principals and teachers reported similar successes and challenges implementing comprehensive 
school reform. The most frequently mentioned successes for both groups were professional 
development, new teaching strategies, and staff commitment to common goals. Principals and 
teachers both mentioned the extra time required to implement new strategies as the most 
significant challenge. The challenges of finding time to collaborate and plan with other staff, 
involving parents, and obtaining buy-in from staff who are reluctant to change also were 
mentioned frequently by both principals and teachers. Teachers who reported on the changes 
they had made in their teaching as a result of comprehensive school reform most often indicated 
using new teaching strategies, incorporating standards and benchmarks into their lesson 
planning, and using more student-centered learning approaches. 

Overall, the CSR Implementation Survey findings reveal a high level of agreement among 
principals and teachers concerning the degree of focus of their reform efforts and the amount of 
progress they had made toward achieving their goals. Geographic locale, grade level, and Title I 
status do not appear to have played a significant role in the reform components that schools 
focused on or in the schools’ goal attainment. School staff reported that the majority of the 
reform components were a primary, as opposed to a secondary, focus of their reform efforts. 
These findings are supported by the results from an in-depth study of 40 CSRD schools in the 
same geographic region (see Faddis et al., 2000). For most kinds of schools, comprehensive 
reform is a slow and difficult process. In spite of the challenges, comprehensive reform also is a 
very rewarding process for those schools whose staff have bought in to the process and are 
willing to give the time needed to make meaningful changes in instructional practices. 
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School ID 



Comprehensive School Reform Implementation — [State] 

Spring 2000 Principal Survey 

The North Central Regional Educational Laboratory (NCREL) is conducting a 7-state study of 
Comprehensive School Reform (CSR). The purpose of the study is to develop a regional 
summary of schools’ progress in implementing their reform plans. We would appreciate your 
participation. Your responses will be confidential and will not be identified with your school or 
district. Survey respondents will receive a copy of the survey report in the summer of 2000. 



Grade Range of School Enrollment Month/Y ear CSR Grant Awarded 



CSR Reform Model(s) You Are Currently 
Using 


Month/Year You Started Working With Model 


1 . 




2. 




3. 





In column A below, please circle a number to indicate the extent to which each area was a focus 
of your comprehensive school reform efforts this school year (1999-2000). 

In column B, please rate the amount of progress you feel your school has made in each area of 
focus. If the area was not a focus, leave column B blank. 



Reform Component 


A 

Extent to Which This is 
a Focus of CSR Efforts 
in 1999-2000 

3- A primary focus 
2- A secondary focus 
1- Not yet a focus 
0- Past focus — goal 
previously addressed 


B 

Rating of Progress in Areas 
of Major or Minor Focus 

5-Goals achieved 
4-Significant progress made 
3-Some progress made 
2-A little progress made 
1-No progress made 


1 . Ensuring that our comprehensive reform plans 
cover curriculum, instruction, assessment, parent 
involvement, professional development, and 
school management, and that these elements are 
appropriately aligned. 


3 2 


1 0 


5 4 3 2 1 


2. Obtaining support from the entire staff in 
implementing the reform models. 


3 2 


1 0 


5 4 3 2 1 


3. Establishing measurable goals and benchmarks 
for student achievement that are aligned with 
state standards. 


3 2 


1 0 


5 4 3 2 1 


4. Involving parents in school reform planning, 
training, or implementation activities. 


3 2 


1 0 


5 4 3 2 1 
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A 








B 






Reform Component 


Extent to Which This is 
a Focus of CSR Efforts 
in 1999-2000 

3- A primary focus 
2- A secondary focus 
1- Not yet a focus 
0- Past focus — goal 
previously addressed 


Rating of Progress in Areas 
of Major or Minor Focus 

5-Goals achieved 
4-Significant progress made 
3-Some progress made 
2-A little progress made 
1-No progress made 








5. 


Using effective, research-based methods and 
strategies for: 




a. Improving curriculum. 


3 2 1 


0 


5 


4 


3 


2 1 


b. Improving instructional strategies. 


3 2 1 


0 


5 


4 


3 


2 1 


c. Improving the integration of technology into 
instruction. 


3 2 1 


0 


5 


4 


3 


2 1 


d. Improving school organization or 
management. 


3 2 1 


0 


5 


4 


3 


2 1 


e. Improving the assessment of student 
progress. 


3 2 1 


0 


5 


4 


3 


2 1 


6. 


Providing professional development for teachers 
to help implement reform strategies. 


3 2 1 


0 


5 


4 


3 


2 1 


7. 


Obtaining additional technical assistance from 
external sources to implement CSR strategies. 


3 2 1 


0 


5 


4 


3 


2 1 


8. 


Using data to evaluate the implementation and 
outcomes of CSR efforts. 


3 2 1 


0 


5 


4 


3 


2 1 


9. 


Obtaining support from the district office to 
implement reform strategies. 


3 2 1 


0 


5 


4 


3 


2 1 


10. 


Ensuring that the CSR-funded reform effort is 
coordinated with our school’s other reform 
efforts. 


3 2 1 


0 


5 


4 


3 


2 1 


11 . 


Coordinating financial resources for CSR with 
other resources to support reform. 


3 2 1 


0 


5 


4 


3 


2 1 



1 . What have been the greatest successes in your CSR efforts to date? 



Thank you for your assistance! 

Please mail your completed survey to 
Larry Friedman, NCREL, 1900 Spring Road, Oak Brook, IL 60521 
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V 



Comprehensive School Reform Implementation — [State] 

Spring 2000 Teacher Survey 

The North Central Regional Educational Laboratory (NCREL) is conducting a 7-state study of 
Comprehensive School Reform (CSR). The purpose of the study is to develop a regional 
summary of schools’ progress in implementing their reform plans. We would appreciate your 
participation. Your responses will be confidential and will not be identified with your school or 
district. Survey respondents will receive a copy of the survey report in the summer of 2000. 




In column A below, please circle a number to indicate the extent to which each area was a focus 
of reform efforts this school year (1999-2000). 

In column B, please rate the amount of progress you feel your school has made in each area of 
focus. If the area was not a focus, leave column B blank. 



Reform Component 


A 

Extent to Which This is 
a Focus of CSR Efforts 
in 1999-2000 

3-A primary focus 
2-A secondary focus 
1-Not yet a focus 
0-Past focus — goal 
previously addressed 


B 

Rating of Progress in Areas 
of Major or Minor Focus 

5-Goals achieved 
4-Significant progress made 
3-Some progress made 
2-A little progress made 
1 -No progress made 


1 . Ensuring that our comprehensive reform plans 
cover curriculum, instruction, assessment, parent 
involvement, professional development, and 
school management, and that these elements are 
appropriately aligned. 


3 2 


1 0 


5 4 3 2 1 


2. Obtaining support from the entire staff in 
implementing the reform models. 


3 2 


1 0 


5 4 3 2 1 


3. Establishing measurable goals and benchmarks 
for student achievement that are aligned with 
state standards. 


3 2 


1 0 


5 4 3 2 1 


4. Involving parents in school reform planning, 
training, or implementation activities. 


3 2 


1 0 


5 4 3 2 1 
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Reform Component 


A 

Extent to Which This is 
a Focus of CSR Efforts 
in 1999-2000 

3-A primary focus 
2- A secondary focus 
1-Not yet a focus 
0-Past focus — goal 
previously addressed 


B 

Rating of Progress in Areas 
of Major or Minor Focus 

5-Goals achieved 
4-Significant progress made 
3-Some progress made 
2-A little progress made 
1-No progress made 


5. 


Using effective, research-based methods and 
strategies for: 


















a. Improving curriculum. 


3 


2 1 


0 


5 


4 


3 


2 1 




b. Improving instructional strategies. 


3 


2 1 


0 


5 


4 


3 


2 1 




c. Improving the integration of technology into 
instruction. 


3 


2 1 


0 


5 


4 


3 


2 1 




d. Improving school organization or 
management. 


3 


2 1 


0 


5 


4 


3 


2 1 




e. Improving the assessment of student 
progress. 


3 


2 1 


0 


5 


4 


3 


2 1 


6. 


Providing professional development for teachers 
to help implement reform strategies. 


3 


2 1 


0 


5 


4 


3 


2 1 


7. 


Obtaining additional technical assistance from 
external sources to implement CSR strategies. 


3 


2 1 


0 


5 


4 


3 


2 1 


8. 


Using data to evaluate the implementation and 
outcomes of CSR efforts. 


3 


2 1 


0 


5 


4 


3 


2 1 


9. 


Obtaining support from the district office to 
implement reform strategies. 


3 


2 1 


0 


5 


4 


3 


2 1 


10. 


Ensuring that the CSR-funded reform effort is 
coordinated with our school’s other reform 
efforts. 


3 


2 1 


0 


5 


4 


3 


2 1 


11. 


Coordinating financial resources for CSR with 
other resources to support reform. 


3 


2 1 


0 


5 


4 


3 


2 1 



1 . What have been the greatest successes in your CSR efforts to date? 



Thank you for your assistance! 

Please mail your completed survey to 
Larry Friedman, NCREL, 1900 Spring Road, Oak Brook, IL 60521 
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